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1.  OBJECTIVES 

Our  goal  has  been  to  develop  a  comprehensive  and  integrated  algorithm  of  processing 
sensor  array  measurements  that  is  ready  for  implementation  on  advanced  sonar  and 
radar  systems.  The  algorithm  should  be  capable  of  detecting,  localizing,  and  tracking 
multiple  signal  sources  with  various  statistical  correlations  and  frequency  band  widths 
in  the  possible  presence  of  array  sensor  uncertainties.  The  algorithm  should  be  robust 
and  proven  to  use  all  the  information  available  in  an  optimal  fashion.  The  following 
issues  have  been  addressed: 

(a)  Exploitation  of  the  linearity  and  separability  structures  in  the  log  likelihood 
function  for  the  fast  and  robust  convergence  of  the  EM  algorithm  and  its 
closely  related  cyclic  regression  (CR)  method. 

(b)  Generalization  of  the  EM  and  the  CR  Methods  to  a  large  class  of  algo¬ 
rithms,  from  which  an  adaptive  algorithm  can  be  obtained  that  has  the  best 
performance  in  all  signal/sensor  environments. 

(c)  Determination  for  the  confidence  regions  of  the  estimated  DOAs  of  the  sig¬ 
nals. 

(d)  T  ection  of  the  number  of  signals  with  a  greater  success  rate  and  less  extra 
computation  than  the  standard  information-theoretic  method. 

(e)  Self-localization  and  calibration  of  the  array  sensors  with  or  without  a  priori 
probability  distribution  of  the  locations  and  phases  of  the  same. 

(f)  Observability  conditions  for  array  sensor  self-localization  and  calibration. 

(g)  Integration  of  narrowband  algorithms  into  a  parallel  algorithm  for  broad¬ 
band  signals. 

(h)  Treating  the  optimal  weighted  subspace  fitting  as  a  weighted  least  squares 
estimation. 

(i)  Extending  the  CR  method  to  maximum  likelihood  retrieval  of  harmonics  in 
colored  noise. 

(j)  Applying  the  CR  techniques  developed  for  array  signal  processing  to  training 
multilayer  perceptrons. 


2.  IMPORTANT  ACCOMPLISHMENTS 


In  order  to  achieve  our  ultimate  goal  stated  in  1,  we  have  emphasized  both  the  the¬ 
oretical  development  and  the  computer  simulation.  All  the  results  to  be  summarized 
below  have  amply  been  simulated  and  verified  in  adverse  signal /sensor  environments. 
The  details  of  the  results  can  be  found  in  the  publication  or  presentation  indicated  at 
the  end  of  each  item.  The  publications  and  presentations  are  listed  in  the  next  Section 
of  this  report. 

(a)  Numerical  acceleration  of  convergence: 

The  EM/CR  type  of  algorithm  is  a  fixed  point  iteration.  Applying  the 
Newton-Raphson  method  to  each  of  the  corresponding  fixed  point  equa¬ 
tions,  we  drastically  speeded  up  the  convergence.  This  idea  of  numerical 
acceleration  was  first  published  by  Steffensen  in  1933.  There  are  many  gen¬ 
eralizations  to  the  multivariable  case,  which  will  be  worth  trying  out  in 
future  research.  [3] 

(b)  Characterization  of  the  rank-n  ambiguity: 

When  the  sensor  locations  have  to  be  estimated  along  with  the  signal  DOAs, 
the  array  manifold  is  parameterized  by  the  phase  differentials.  It  may  inter¬ 
sect  the  signal  subspace  at  points  that  represent  false  action  vectors.  This 
phenomenon  was  called  rank-n  ambiguity  by  R.O.  Schmidt. 

A  necessary  and  sufficient  condition  for  the  rank-2  ambiguity  is  given,  whose 
geometric  meaning  as  well  as  an  example  are  examined  in  detail.  We  also 
offered  a  conjecture  about  the  general  rank-n  amgibuity.  Its  proof  or  disproof 
is  an  open  problem.  [4] 

(c)  Characterization  of  the  phase  delay  ambiguity: 

The  phase  differentials  are  taken  modulo  27 r  in  the  arguments  of  the  ac¬ 
tion  vector  components,  resulting  in  the  so-called  phase  delay  ambiguity. 
Geometrically,  this  translates  into  an  ambiguity  concerning  the  relative  dis¬ 
tances  between  the  array  sensors  along  each  signal  DO  A.  In  other  words, 
the  relative  distances  can  only  be  determined  modulo  an  integral  multiple 
of  each  signal  wavelength. 

The  phase  delay  ambiguity  can  be  avoided  by  either  starting  with  very  good 
a  priori  sensor  location  estimates  or  using  known  and  unknown  calibrating 
signals  with  certain  frequency/geometry  structure.  A  necessary  and  suffi¬ 
cient  condition  is  given  for  no  phase  delay  ambiguity.  [4] 

(d)  Minimal  number  of  known  and  unknown  signals  for  sensor  location  observ¬ 
ability: 
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It  was  observed  by  Rockah  and  Schultheiss  that  the  shape  of  the  sensor  array 
is  observable,  provided  that  three  signals  with  unknown  but  different  DOAs 
are  available.  Obviously,  if  there  are  no  known  signals  in  the  measurements, 
a  rotation  or  reflection  of  the  array  can  not  be  observed  from  unknown  signals 
alone,  no  matter  how  many.  We  examined  in  detail  the  numbers  of  known 
and  unknown  signals  required.  The  assumptions  are  made  that  the  phase 
delays  are  available  without  ambiguity  and  the  measurements  are  noise-free. 

[4] 

(e)  A  large  class  of  cyclic  regression  algorithms  for  multiple  signal  direction 
finding: 

In  each  step  in  the  iteration  cycle,  the  standard  CR  maximizes  the  log- 
likelihood  function.  But  in  the  step  of  estimating  the  action  vectors,  the 
covariance  matrix  of  the  signals  needs  to  be  inverted.  Since  the  covariance 
matrix  of  coherent  signal  is  not  invertible,  the  standard  CR  without  ridge 
regression  fails  to  work  for  coherent  signals. 

The  EM  algorithm  circumvents  this  difficulty  by  evaluating  the  conditional 
expectation  of  the  action  vectors  given  the  sensor  measurements.  However, 
it  converges  slowly. 

Some  gradient  descent  methods  were  applied  to  deal  with  the  same  difficulty. 
The  performances  of  the  resulting  algorithms  turned  out  to  be  very  good  for 
some  signal/noise  situations  and  very  poor  for  others. 

A  close  examination  of  the  equations  for  these  algorithms  revealed  that  they 
all  belong  to  the  same  class  with  different  values  of  a  class  index.  The  index 
can  be  viewed  as  the  proportions  of  the  measurement  noise  variance  they 
are  assigned  to  each  piece  of  complete  data,  namely  the  action  vector  times 
the  signal  waveform. 

Five  algorithms  from  the  class,  each  with  some  intuitive  backing,  were  sin¬ 
gled  out  for  simulations  study.  Results  are  very  interesting  and  greatly 
enhance  our  understanding  of  the  algorithms.  Nevertheless,  an  adaptive  al¬ 
gorithm  that  works  best  in  all  situations  has  yet  to  be  found.  More  statistical 
analysis  needs  to  be  done.  [6] 

(f)  Array  sensor  localization  and  calibration  by  cyclic  regression: 

Our  method  is  an  extension  of  the  cyclic  regression  for  direction  finding. 
The  extension  keeps  the  essential  characteristics  of  cyclic  regression,  namely 
the  iteration  over  a  finite  number  of  linear  regression  stepts.  In  addition  to 
the  separability  and  the  linearity  structures  used  for  direction  finding,  the 
extension  is  made  possible  by  the  property  that  the  sensor  coordinates  and 
phases  enter  the  arguments  of  the  complex-valued  components  of  the  action 
vectors  in  a  linear  fashion. 
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A  salient  feature  of  our  method  as  opposed  to  others  is  that  direction  find¬ 
ing  and  sensor  localization  and  calibration  can  be  done  at  the  same  time 
on  the  same  set  of  measurements,  as  long  as  the  observability  conditions 
for  all  the  unknown  parameters  are  satisfied.  This  is  important,  when  the 
fluctuations  of  the  sensor  locations  and  phases  are  significant  from  set  to  set 
of  measurements.  [5] 

(g)  Integration  of  the  a  priori  information  about  the  signal  DOAs  and  the  sensor 
locations  and  intrinsic  phases: 

Two  methods  of  integrating  the  a  priori  information  into  our  maximum 
likelihood  estimation  for  signal  direction  finding  and  sensor  self-localization 
and  calibration  have  been  obtained. 

• 

If  the  a  priori  information  is  highly  unreliable  and  the  observability  con¬ 
ditions  are  satisfied  modulo  an  unknown  rotation,  we  first  determine  the 
shape  of  the  sensor  array  using  the  sensor  measurements  only  and  then  ro¬ 
tate  the  sensor  array  with  the  fixed  shape  to  fit  the  a  priori  information.  An 
interesting  criterion  for  rotation  has  been  derived. 

If  the  a  priori  information  is  reliably  described  as  probability  densities  and 
the  observability  conditions  are  satisfied  (perhaps  with  the  help  of  the  a 
priori  information),  the  cyclic  regression  is  easily  modified  to  integrate  the 
priori  information  by  augmenting  the  linear  regression  equations  in  each 
iteration  cycle.  [6] 

(h)  In  a  recent  paper,  Viberg  and  Ottersten  proposed  a  weighted  subspace  fitting 
(WSF)  criterion  for  direction  finding.  However  their  algorithm  for  WSF 
involves  the  computation  of  the  eigenvalues  and  eigenvectors  of  the  sample 
covariance  matrix  and  a  Gauss-Newton  type  algorithm.  We  (P.I.,  Nagaraj 
and  Rukhin)  derived  an  asymptotically  equivalent  expression  of  the  WSF 
criterion  which  does  not  involve  eigenvalues  and  eigenvectors.  Facilitated 
with  the  expression,  a  CR  algorithm  was  obtained  to  replace  the  Gauss- 
Newton  type  algorithm  for  estimating  the  signal  DOAs.  [7] 

(i)  The  estimation  of  the  frequencies  of  sinusoidal  components  embedded  in 
white  or  colored  noise  is  one  of  the  fundamental  problems  encountered  in  a 
large  range  of  fields.  We  succeeded  in  extending  the  CR  method  to  evaluating 
the  maximum  likelihood  estimates  of  these  component  frequencies.  [8] 

(j)  Due  to  their  almost  unlimited  capability  to  represent  nonlinearities,  the 
multilayer  perceptrons  (MLPs)  have  found  a  wide  range  of  applications  in¬ 
cluding  signal  processing  and  adaptive  control.  Finding  a  fast  and  accurate 
method  of  training  an  MLP  has  been  a  focal  point  of  research  in  the  field 
of  artificial  neural  networks.  At  least  10  different  training  algorithms  have 
been  published  in  the  past  5  years. 
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The  application  of  the  CR  techniques  to  the  training  problem  yielded  a 
novel  training  algorithm,  which  we  call  the  backestimation  algorithm.  As 
opposed  to  the  standard  backpropagation  of  “deltas”,  which  are  actually  the 
gradients  of  the  error  surface,  we  backestimate  the  neuron  outputs.  More 
specifically,  we  compute  the  probabilistic  expectation  of  the  output  of  a  layer 
conditioned  on  that  of  the  next  layer. 

Results  of  extensive  numerical  experiments  show  that  both  the  convergence 
rate  and  the  accuracy  level  of  backestimation  are  better  than  those  of  back- 
propagation  and  its  variants  by  several  orders  of  magnitude.  [9] 

3.  SIGNIFICANCE 

If  I  am  asked  to  summarize  all  my  results  in  one  sentence,  I  would  say  that  they  are 
simply  a  very  effective  decomposition  of  the  nonlinear  least-squares  estimation  problem 
into  linear  regressions.  The  decomposition  enables  us  to  handle  information  from  dif¬ 
ferent  times,  frequencies,  and  sources  in  an  efficient  and  integrated  manner.  Although 
we  have  concentrated  on  the  maximum  likelihood  approach,  which  is  a  nonlinear  least- 
squares  problem,  our  results  are  readily  applicable  to  other  least-squares  formulations 
such  as  the  optimal  weighted  subspace  fitting. 

(a)  The  techniques  developed  for  integrating  the  a  priori  information  in  the  CR 
algorithm  are  of  particular  importance.  They  suggest  a  recursive  algorithm 
for  maximum  likelihood  direction  finding  in  the  presence  of  array  sensor 
uncertainties.  A  recursive  algorithm  is  desired,  if  the  signal  source  and/or 
the  array  sensors  are  mobile  or  if  the  best  possible  signl  DOA  estimates 
are  constantly  required.  The  recursive  algorithm  updates  the  estimates  for 
each  in-coming  snapshot.  It  should  take  full  advantage  of  all  the  current 
information  in  terms  of  the  current  estimates  and  their  covariances  and  the 
newly  acquired  snapshot.  The  P.I.  is  working  on  such  a  recursive  algorithm. 

(b)  The  discovery  of  a  large  class  of  CR  algorithms  with  specific  class  index 
parameters  greatly  enhanced  our  understanding  of  the  structures  of  cyclic 
regression.  It  calls  for  an  adaptive  way  of  tuning  those  index  parameters  so 
that  the  CR  algorithm  adaptively  works  best  in  all  signal/sensor  environ¬ 
ments. 

(c)  The  transformation  of  Viberg  and  Ottersten’s  WSF  criterion  and  the  use  of 
a  CR  algorithm  for  minimizing  it  greatly  reduce  the  amount  of  computation 
needed  to  implement  the  WSF  for  direction  finding. 

(d)  The  CR  algorithm  for  the  retrieval  of  harmonics  in  colored  noise  eliminates 
the  following  three  difficulties  associated  with  existing  methods: 
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(1) .  White  noise  assumption:  Very  often  the  Nyquist  sampling  rate  disallows 

sufficient  decorrelation  between  measurements  required  for  the  white 
noise  assumption. 

(2) .  Time- aperture  reduction:  The  CR  algorithm  does  not  require  the  esti¬ 

mation  of  correlations  and  thus  avoids  time-aperture  reduction. 

(3) .  Excessive  computation  associated  with  Gauss-Newton  procedure. 

(e)  The  backestimation  algorithm  for  trainin  multilayer  perception  has  the  po¬ 
tential  to  greatly  outperform  the  standard  backpropagation  and  its  variants. 
It  is  based  on  the  cyclic  regression  techniques  and  thus  inherits  the  versatility 
of  the  CR  algorithms. 

4.  FUTURE  RESEARCH 

Many  problems  have  been  resolved  during  the  contract  period  for  developing  a  com¬ 
prehensive  and  integrated  array  signal  processing  system  based  on  the  CR  algorithm. 
Three  major  tasks  remain  to  be  carried  out: 

(a)  Develop  a  recursive  algorithm  for  maximum  likelihood  direction  finding  in 
the  presence  of  array  sensor  uncertainties. 

(b)  Derive  a  signal  detection  scheme  that  works  jointly  with  the  CR  algorithm 
so  as  to  achieve  a  greater  success  rate  and  less  extra  computation. 

(c)  Integrate  the  narrowband  CR  algorithms  into  a  parallel  algorithm  for  broad¬ 
band  signals. 
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